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Abstract 

For a closed-loop system, which has a contention-based multiple access network on its sensor link, the Medium Access 
Controller (MAC) may discard some packets when the traffic on the link is high. We use a local state-based scheduler to select 
a few critical data packets to send to the MAC. In this paper, we analyze the impact of such a scheduler on the closed-loop 

^vj system in the presence of traffic, and show that there is a dual effect with state-based scheduling. In general, this makes the 

optimal scheduler and controller hard to find. However, by removing past controls from the scheduling criterion, we find that 
certainty equivalence holds. This condition is related to the classical result of Bar-Shalom and Tse, and it leads to the design of 
a scheduler with a certainty equivalent controller. This design, however, does not result in an equivalent system to the original 
problem, in the sense of Witsenhausen. Computing the estimate is difficult, but can be simplified by introducing a symmetry 
constraint on the scheduler. Based on these findings, we propose a dual predictor architecture for the closed-loop system, which 
ensures separation between scheduler, observer and controller. We present an example of this architecture, which illustrates a 

Mh network-aware event-triggering mechanism. 
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Tjy I- Introduction 

f^i ONSIDER a network of control systems, where the communication between the individual sensors and controllers of 

^^ different control loops occurs through a shared network, as shown in Fig. [T] This is an important scenario, in the context 

of wireless Networked Control Systems (NCS), for industrial and process control |T|. A medium access control layer is 

required in the sensor's protocol stack to arbitrate access to the shared network. To focus on the implications of a Medium 

Access Controller (MAC) on the sensor link, we assume that the communication between the controllers and the corresponding 

actuators occurs over a point-to-point network, not a shared network. This is a common architecture, in practice Q, Q. The 

f-j MAC can implement a contention-free or a contention-based multiple access method, both of which have their own challenges 

^^ lU. A contention-free multiple access method requires a dynamic scheduler to prevent poor channel utilization, and such a 

-^T scheduler is hard to construct and implement on an interference-constrained shared network [51, 0. Contention-based methods 

^^ have proven popular in standards such as IEEE 802.15.4 (7), as they facilitate an easy deployment on sensor nodes. However, 

C\| such methods result in random access, which could significantly deteriorate the performance of a closed-loop system [8|. Thus, 

the design of a MAC for networked control systems is a challenging problem, and calls for innovative solutions 0. 

^ In this paper, we explore the design of a state-aware contention-based MAC, as opposed to an agnostic contention-based 

k> MAC. The state-aware MAC is capable of influencing the randomness of channel access in favour of the state of the plant in 

rN the closed-loop system. However, directly using the state of the plant to determine an access probability may result in a MAC 

that is difficult to implement and analyze ifTOl . Instead, we use the state of the plant to select packets to send to the MAC, 

motivated by an understanding of the two roles played by a MAC: Any random access method works by resolving contention 

between simultaneous channel access requests, thus spreading traffic that arrives in bursts. The carrier sense multiple access 

with collision avoidance (CSMA/CA) method does this by assigning a random back-off to packets that attempt to access a 

busy channel, thus spreading the traffic over a longer interval of time. Similarly, the p-persistent CSMA method does this by 

probabilistically limiting access to the channel and permitting a number of retransmissions if the channel is busy |11|. However, 

all of these methods permit only a finite number of retransmissions, beyond which the packet is discarded. We appropriate this 

latter role of discarding packets to a local state-based scheduler, which sends fewer, but more important packets to the MAC 

for transmission across the network. 

A similar strategy has previously been proposed from the more general perspective of reducing network traffic |3|. When 
applied to the newly posed NCS problem [12], this approach has driven the design of event-based sampling systems [ 131 , lfl4l . 
which have been shown to outperform periodically sampled systems under certain conditions |[T5l , |[T6lL ifTTl . We approach 
the same problem from a different perspective, but one that leads to a network-aware design of event triggering methods. 

There are two main contributions in this paper. The first contribution is an analysis of the impact of having a state-based 
scheduler in the closed-loop. Primarily, a state-based scheduler permits the information available at the controller to be altered 

C. Ramesh, H. Sandberg and Karl H. Johansson are with the ACCESS Linnaeus Centre, Electrical Engineering, KTH Royal Institute of Technology, 
Stockholm, Sweden, e-mail: ({cramesh,hsan,kallej}@ee.kth.se). 



x'' r pO 



py 



C (1) 



M) 



<S (1 




(KM) 



p(1) 



£(M) 



p(M) 



pO) 


j th Plant 


C (j) 


j th Controller 


5® 


i th Source 


p(i) 


i th Destination 


A/" 


Network 


■■■■► 


Sensor Link 


— 


Actuator Link 



Fig. 1. A network of M control loops, with each loop consisting of a plant V^ and a controller CO') for j £ {1, . . . , M}. The loops share access 
to a common medium on the sensor link, along with N other communication flows from generic source-destination pairs. The controllers and actuators 
communicate over dedicated networks, not shared links. 

with the plant state. This information is not entirely random, like in the case of packet losses due to a noisy channel (T8) , (19) , 
and it can result in a sharply asymmetrical estimation error, unlike in the case of encoder design over limited data rate channels 
1 20], 1 21 ]. It seems reasonable to ask if we can use the controller to move the plant state across the threshold and force a 
transmission? If this is possible, the controller plays two roles: the first one being to control the plant, and the second one being 
to control the information available at the next time step. This relates to the classical concept of a dual effect, as described in 
1 22 1, [23]. The answer to this question determines the ease of optimal controller design, as the Certainty Equivalence Principle 
would not hold if there is a dual effect [24]. We examine our system and find that there is a dual effect with a state-based 
scheduler in the closed-loop, and thus, that the certainty equivalence principle does not hold. Hence, the optimal state-based 
scheduler, estimator and controller designs are coupled. A restriction on the input arguments to the state-based scheduler, such 
that these arguments are no longer a function of the past control actions, renders the setup free of a dual effect, and enables 
the certainty equivalence principle to hold. These results can be seen as an interpretation, within the state-based scheduler 
setup, of the classical work on information patterns [25], dual effect, certainty equivalence and separation by Witsenhausen 
(26), Bar-Shalom and Tse (24), and on adaptive control by Feldbaum (22) . Astrom and Wittenmark [23] and others [27]. 

The second contribution of this paper is on the dual predictor architecture, which is our proposed solution to the state-based 
scheduler design problem. In this architecture, the state-based scheduler thresholds the squared difference of the innovation 
contained in the latest measurement to the estimator across the network. This results in an optimal certainty equivalent controller, 
and a simple observer which generates the minimum mean-squared error (MMSE) estimate. Tuning parameters in the state-based 
scheduler in this architecture based on the current network traffic could result in a scheduling law that guarantees a probabilistic 
performance. This is not easy to show, in general, as the performance analysis of a closed-loop system with a state-based 
scheduler in a multiple access network is a difficult problem (16), (T7) . However, we illustrate the guaranteed performance using 
simulations, and thus claim that the state-based scheduler we propose results in a network-aware event-triggering mechanism. 

The rest of the paper is organized as follows. In Section [il] we present the problem formulation. In Section III we derive 
theoretical results for the case when full state information is available, with and without exogenous network traffic. In Section IV 
we present the dual predictor architecture. We look at an extension to output-based systems in Section [V] We present an 
example, which illustrates our notion of network-aware event- triggering, in Section [VTI Providing performance guarantees 



remains a difficult problem, as we indicate under future work, along with the conclusions, in Section |VIII[ 



w 



II. Preliminaries 

E present the problem setup and a few important definitions, along with a review of the classical concepts of dual 
effect and certainty equivalence in this section. 



A. Problem Formulation 



We consider a network of M control loops, as shown in Fig. 2(a) Each control loop, for j E {1, . . . , M}, consists of a 
plant Vv\ a state-based scheduler So) and a controller Co). The loops share access to a common medium on the sensor link. 



A closed-loop system in this network can be modelled as shown in Fig. 2(b) with the index j dropped for simplicity. The 
block J\f represents the network as seen by this loop, and the block 1Z denotes the Contention Resolution Mechanism (CRM), 



which determines access to the network. Each of the blocks in Fig. 2(b) is explained below. 
Plant: The plant V has state dynamics given by 

Xk+i = Ax k + Bu k + itffc, 

where A eR nxn , B Gt nxm and w k is i.i.d. zero-mean Gaussian with covariance matrix R w . The initial state xq is zero-mean 
Gaussian with covariance matrix Rq. 
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(a) A multiple access (MA) scenario for NCSs 



"* 


V 


x k 




s 






/ 


■ --->> 


<: 


n 












/Vr 


Af 




c 


yk 

< — 




°k\ 










AchT 

















(b) The MA model for each closed-loop system 



Fig. 2. A plant (V^), state-based scheduler (<S^) and controller (C^) share the network (AT) with M — 1 other closed-loop systems with 
state-based schedulers (S^\j G {2, . . . , M}), and iV generic sources (S^\i G {1, . . . , iV}), in (a). A model, from the perspective of a 
closed-loop system in the network, is depicted in (b). 



State Based Scheduler: There is a local scheduler <S, situated in the sensor node, between the plant and the controller, which 
decides if the state is to be sent across the network or not. The scheduler output is denoted 7&, where 7^ G {0, 1}. It takes a 
value 1 when the state Xk is scheduled to be sent and otherwise. The scheduling criterion is denoted by the policy /, which 
is defined on the information pattern of the scheduler I k , and is given by 



Ik = fk(Uo \0J S k ) , 



(2) 



where, fk is a non-trivial function of Uq 



G fi|, and Q s k is the sigma-algebra generated by the information set at 



the scheduler, given by I k = {xj, 2/o _1 ,7o _1 , ^o -1 }- Here, we use bold font to denote a set of variables such as aj = 

{a t , CLt+i, • • • , «t}- Note that an explicit acknowledgement (ACK) of a successful transmission is required for 5k to be available 

to the scheduler. 

Network: The network J\f generates exogenous traffic, as is indicated by rik G {0, 1}. It takes a value 1 when the network 

traffic attempts to access the channel, and otherwise. The network traffic is considered to be stochastic, as it could be 

generated by another such control loop, or by any other communicating node in the network. Thus, rik is a binary random 

variable, which is not required to be i.i.d. We say that there is no exogenous network traffic if rik = 0, for all k. 

CRM: The CRM block 1Z resolves contention between multiple simultaneous channel access requests, i.e., when 7^ = 1 and 

rik — 1. If the CRM resolves the contention in favour of our control loop, 5& = 1, and otherwise 0. The CRM can be modelled 

as the MAC channel response 1Z, with MAC output Sk given by 



5 k =^(7 fc ,n fc ) 



(3) 



For brevity, we also define 5k = 1 — 5k, which takes a value 1 when the packet is not transmitted. The MAC channel response 

1Z is modelled as a discrete memoryless channel at the sampling time scale, requiring the CRM to resolve contention with 

respect to this packet before the next sampling instant. This translates to a limitation on the sampling rates supported by the 

model. 

Measurement: The measurement across the network is denoted yk- It is a non-linear function of the state Xk, and is given by 



Vh = hxk = 




(4) 



A successful transmission results in the full state being sent to the controller. However, even non-transmissions convey 
information as the scheduler output 5k can be treated as a noisy and coarsely quantized measurement of the state Xk • 
Controller: The control law g denotes an admissible policy for the finite horizon N defined on the information pattern of 
the controller, I k , and is given by 

Uk = 9k(u c k ) , 

where, uj k G Q k , and Q k is the sigma-algebra generated by the information pattern I k 
function, defined over a horizon TV is given by 

N-l 



(5) 
{ Hq , #o , Uq ~ x } . The obj ective 



J(/,fl)=E 



xJjQoXN + ^2 ( x lQl x s + u lQ2U s ) 



(6) 



where Qo and Q\ are positive semi-definite weighting matrices and Q2 is positive definite. 



In the rest of the paper, we address the following questions - 

1) For a NCS with a state-based scheduler, what is the optimal control policy which minimizes the cost J in ([6])? 

2) Can we find a simple, but sub-optimal, closed-loop system architecture for the given NCS? 

To answer the first question, we need to examine whether the system exhibits a dual effect. This also requires us to check if 
we can find an equivalent system, in the sense of Witsenhausen, for which certainty equivalence holds. The second question 
requires us to identify restrictions on the scheduling policy /, which can ensure separation of the scheduler, controller and 
observer. 

B. Definitions and Properties 

We present a few definitions and properties that are used in the rest of the chapter. 

Definition 2.1 (Uncontrolled Process): An auxiliary uncontrolled process (P) can be defined for any closed-loop system, by 
removing the effect of the applied control signals from the state. The resulting uncontrolled state is denoted x k , and given by 

k k 

x k =x k ~Y^ A £ - 1 Bu k - i = A k x + Yl Ai ~ lw k-£ • (7) 

£=1 £=1 

Last Received Packet Index: The time index of the last received packet is denoted r k at time k (illustrated in Fig. 3(a)[ ), and 
is given by 

Tfc = max{£ : S t = 1, for — 1 < t < k} and <5_i = 1, — 1 < r k < k (8) 

An iterative relationship for r k can be found as 

Tk = hVs-i + fak, r_i = -1 (9) 

If a packet arrives at current time k, the last received packet index r k = k. But, if there is no packet at time k, then the 
last received packet index is the same as the last received packet index from time k — 1, i.e., r k = r k -i. This implies that 

Dual Effect: Note that the control u k might affect the future state uncertainty, in addition to its direct effect on the state. 
This is called the dual effect of control |22], and is discussed for state-based schedulers in Section |IlI-A[ 
Definition 2.2 (No dual effect [24]): A control signal is said to have no dual effect of order r > 2, if 

E[Myf k ] = E[M^\x , w? , ng] (10) 

where M ki = (x k ,i —E[x k ^\I k ]) r is the r th central moment of the i th component of the state x k ^ conditioned on I k and r k 
is the time index of the last received measurement at time k. 



Note that M ki in (10) must specifically not be a function of the past control policies g^ 1 for the control signal to have no 
dual effect of order r. In other words, if there is no dual effect, the expected future uncertainty is not affected by the controls 
Uq' 1 . In the presence of a dual effect, the optimal control laws are hard to find l23ll . 

Certainty Equivalence: There are two closely related terms: a certainty equivalent controller and the Certainty Equivalence 
Principle. We define both these terms with respect to the deterministic optimal controller, with full state information, for the 



above problem setup |28], [24]. These properties are discussed for state-based schedulers in Section [III-C 

Definition 2.3 (Certainty Equivalent Controller): A certainty equivalent controller uses the deterministic optimal controller, 
with the state x k replaced by the estimate Xk\k = E[x k \I k ], as an ad hoc control procedure. 

Sometimes, there is no loss in optimality in using a certainty equivalent controller. Then, we say that the Certainty Equivalence 
Principle holds. 

Definition 2.4 (Certainty Equivalence Principle): The Certainty Equivalence Principle holds if the closed-loop optimal con- 
troller has the same form as the deterministic optimal controller with the state x k replaced by the estimate Xk\k. 

Correlated Network Noise: We state a property of feedback systems with a state-based scheduler which share a contention- 
based multiple access network. Even if the initial states and disturbances of all the plants in the network are independent, the 
contention-based MAC introduces a correlation between the traffic sources in the network, as noted in lfT6lL ifTTl . 

Lemma 2.1: For a closed-loop system defined by ([T])-([5]), the exogenous network traffic indicated by n k is correlated to the 
state of the plant x k . 

Proof: The MAC output 5 k _i is a function of the state x k -i and the indicator of network traffic n k _i, from ^ and 
([5]). The control signal u k -i is a function of the MAC output S k -i from ([5]), and is applied through feedback to the plant. 
Thus, Xk and j k are correlated to S k -i. Similarly, the network traffic from other closed-loop systems (and its indicator n k ) is 
correlated to 5 k -\, and consequently, x k . ■ 
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Fig. 3. An illustration of the delay since the last received packet (d k ) and the index of the last received packet (r k ) in (a). In (b), the shaded area represents 
the non-scheduled region for a state x G I 2 , as defined by the scheduling policy /. States in this region are not considered to be events and correspondingly 
result in a non-zero estimation error. Outside this region, the estimation error is zero. Thus, the controller has an incentive to move some states along path 2, 
as compared to path 1. 



III. Optimal Controller Design 

WE present the main results of this paper in this section. We first analyze the effects of a state-based scheduler on a 
control loop with no exogenous network traffic, i.e., n k = 0. As a consequence of this, the MAC output is equal to 
the scheduler output, i.e., 5 k — 7&. We show that there is a dual effect of the control signal, and that the scheduling policy 
must be restricted from using the past control inputs for the Certainty Equivalence Principle to hold. We illustrate this for a 
second order system with a state-based scheduler in Fig. |3(b)| and show that the controller is not oblivious to the scheduler 
boundaries. We extend our results to the case with exogenous network traffic. 



A. Dual Effect with State-based Scheduling 



For the problem defined in Section II-A we observe the following result. 

Theorem 3.1: For the closed-loop system defined by ([T])-([5]), with no exogenous network traffic, the control signal has a 
dual effect of order r = 2. 

Proof: We examine the estimation erro r, and show that it is not equivalent to the estimation error generated by the 
uncontrolled process (V from Definition 2.1 ) in the place of the plant V . Thus, we proove that the estimation error covariance 
is a function of the applied controls ul' 1 . 

From ([4]), we know that a successful transmission results in the full state being sent to the controller, whereas a non- 
transmission conveys only a single-bit of information (5 k is binary) about the state to the controller. Thus, the estimate, 
Xk\k = E[#fc|I fe ], is given by 

Xk\h =5 k x k + 6 k E[x k \I k ,5 k = 0] . 

This estimate always depends on S k , due to the asymmetry in the resolution of the received information with and without a 
transmission. The scheduler outcome, and consequently S k , are influenced by the applied control inputs Uq' 1 in a state-based 
scheduler such as d2j). 



The estimation error, defined as Xk\k = x k — E[x k \I k ], is given by 
x k \k = (x k - E[x k \I k , 5 k = 0]) • 5 k , 



(11) 



and can also be seen to depend on 5 k . The estimation error when there is no transmission is defined as x° klk = x k — E[x k \I k , 5 k 
0], and is given by 



°k\k 



A k x Q + Y^A i - 1 {B> 



u k -£ + w k - £ 



) - E[A k x + J2 A £ -\Bu k _ £ + w k - £ )\lZ,5 k = 0] 



x k 



-E[x k \I k ,5 k = 0], 

where x k is the state of the uncontrolled process (see Definition |2.1| ). Thus, the first term of the product in (11) remains 
unchanged if the plant is replaced by the uncontrolled process. However, the effect of the applied controls cannot be removed 
from 5 k = (1 — 5 k ) 9 which is the second term of the product in ( 11 ). 

Thus, the estimation error is always dependent on the applied controls and this distinguishes the current problem from other 
related problems, such as in l20lL l2T1l . The error covariance, Pk\k = E[xk\kX^ k \I k ], is given by 



Pk\k = 5 k • (E[xk\kxl lk \I k , 5 k = 0]) . 



(12) 



The covariance Pk\k is zero if the scheduling criterion in ^ is fulfilled, and non-zero otherwise. Clearly, Pk\k is a function of 
the past controls. Hence, Pk\k does not satisfy the condition ( [TO] ) required to have no dual effect. Thus, we see that the system 
([T])-([5]) exhibits a dual effect of order r = 2. ■ 



In this setup, there is an incentive for the control policy to modify the estimation error along with controlling the plant, as 



illustrated in Fig. 3(b) Thus, the controller might prefer to move some states along path 2, as compared to path 1, to improve 



the estimation error. 

B. Equivalent Schedulers vs. Equivalent Systems 

Every state-based scheduler /, defined in (|2J), can be transformed into an equivalent scheduler /, such as 

Ik = f k (u,' k ) , (13) 

where, Co k G ft s k , and £l s k is the sigma-algebra generated by the information pattern I k = {xo^Wq' 1 }. The applied controls 
ul' 1 can be reconstructed at the scheduler at time k, and hence, such a transformation can always be accomplished. We now 
examine the question of whether the closed-loop system with this equivalent scheduler, is equivalent to the original system. 
Witsenhausen [26] defines an equivalent design, which gives us the following definition when applied to our problem. 

Definition 3.1: An equivalent (in the sense of Witsenhausen) control design g eq for the optimal controller g* , which minimizes 
the cost criterion ^ for the system defined by ([T]h[5]), satisfies the equivalence relationship given by 

u*=T(u;,<r) = T(u>,£e q ), (14) 

where T is obtained by recursive substitution for the control signals in the system equations with the respective control policy 
and the primitive random variables u^ = [xq^Wq' 1 ]. 

For brevity, we adopt the following notation. Let {V,fi,gi} denote a system with the plant given by {!]), with fi as the 
given scheduler and g\ as the optimal controller for the cost criterion ^ applied to this system. We now note the following 
result. 

Theorem 3.2: For two schedulers /, given by (|2J), and /, given by |l3J), which result in the same schedules for the closed- 
loop system given by |l|, (|3j)-([5j), with no exogenous network traffic, {V,f,g} is not an equivalent system to {V,f,g*}, in 
the sense of Witsenhausen. 



Proof: Definition |3 . 1 1 requires the control signals obtained using the policies g* and g to be equal. In this proof, we find 
the optimal control policies for {V, f,g} and {V, /,#*}, and show that they do not result in the same control signals. 

For the optimal control policy, which minimizes the quadratic cost J in ([6]), to be certainty equivalent, we need to find a 
solution to the Bellman equation [28], which is a one-step minimization of the form 

V k = min ^[xlQix k + ulQ 2 u k + V k +i \l k } . (15) 

In general, without defining a structure for the estimator, the solution to the functional is given [24] in the form of 



V k =E 



x k S k x k \I k 



s k , (16) 



where S k is a positive semi-definite matrix and both Sk and s k are not functions of the applied control signals uS _1 . We now 
prove that a solution of this form can be found for {V,f,g}, but not for {V,f,g*}. 

First consider the system {V,f,g}. We denote the state and control signals of this system as x k and u k . At time N, the 
functional has a trivial solution with Sn = Qo and sn = 0. This solution can be propagated backwards, in the absence of a 



dual effect. To show this, we use the principle of induction, and assume that a solution of the form ( [16] ) holds at time k + 1. 
Then, at time k, we have 



V k = min ElxlQtXk + ulQ 2 u k + x k+1 S k+1 x k+1 + s/c+iK] 

Uk 

= min E[x£(Qi + A T S k+1 A)x k \f k ] + ti{S k+1 R w } + E[s k+1 \f k ] 

Uk 

+ ul(Q 2 + B T S k+1 B)u k + xl\ k A T Sk+iBiik + ulB T S k+1 Ax k \ k 



rr c 1 



where Xk\k =E[x k \I k ]. The optimal control is found to be 

u k = -L k x k \ k , L k = (Q 2 + B T S k+1 B)~ 1 B T S k+1 A . (17) 

Substituting the expression for u k into V k gives us a solution of the form in ( [T6] ), with 

S k = Qi + A Sk+iA — A S k +iB(Q 2 + B S k +iB)~ B S k +iA , 

s k = tr{S fe+ ii^}+E[s fe+1 |4]+t^ (18) 

where the matrix Sk is positive semi-definite and not a function of the applied controls ul' 1 . The scalar Sk is not a function 
of the applied controls ul' 1 if and only if P k \k has no dual effect [24]. From the expression for the error covariance Pk\k 
( [T2] ), it is clear that a scheduling criterion that is not a function of the past control actions, such as ( [T3] ), results in no dual 



effect. Under this condition, s k is not a function of the applied controls Uq' 1 and the proof by induction is complete. Since 

the Certainty Equivalence Principle holds. 
Solving the backward recursion as we did above, we 

- Qq and sn = 0, and Sn-i and s/v-i given by ([18]) 



the optimal control signal (17) is a function of only the estimate Xk\k, 
Now, consider the system {V,f,g*} with state x k and control u^. 
find that Vn and Vjv-i have a solution of the form (16), with Sn - 
with k = N — 1. However, Vat-2 results in a different minimization problem for this system because of the dual effect in 
{V, /, #*}, as indicated next. The optimal control signal u* N _ 2 can be obtained by solving 

dV N _ 



du 



= 2u*x_ 2 (Q 2 - 



B 1 S N - 1 B)+2x 



N-2\N- 



2 A'S 



N-l 



B 



N-2 



d 



du\ 



N-2 



ti{A T S N B(Q 2 + B T S N B)~ 1 B T S N A ■ E[Pi 



N-1\N- 



|IiV- 2 ]} 



Multiplying the above expression with (Q 2 + B T Sn-iB)~ 
we obtain the simpler equation 

dV N - 



1 from the right and using ( 17 ) to denote vf^ E _^ 



jN— 2 Xn-2\n- 



du* N _ 2 

where, we set Kat-2 = (Q2 
estimation error covariance Pi 



2(«f_ 



u 



CE,T 

N-2 



) + 



du\ 



N-2 



tr{Kiv-2 E[Pj 



N— 1\N- 



^N-2}} =0 



(19) 



B T S N - 1 B)- 1 A T S N B(Q 2 



B T S N B)- 1 B T S N A. 



The last term in (19), related to the 



AT— 1|JV- 



is not equal to zero as implied by the dual effect property from Theorem |3.1| Due to 



this term, the above minimization problem is not linear, and thus, the solutions 1^-2 anc * u n-2 are not ec l u al. Since u^_ 2 
has the same form as un-2, we also note that un- 



2 and u* N _ 2 have very different forms. From this point on, the cost-to-go 



for the optimal control policy g* does not have a solution of the form given by (16). Hence, the control signals {u}o~ 3 and 
{u*}o~ 3 will not be equal. Now, the joint distribution of all system variables could be quite different for schedulers / and /. 
Thus, the described transformation of the scheduling criterion does not result in an equivalent class construction. ■ 

Due to the dual effect, the optimal control action takes on two roles. One, to control the plant, and the other, to probe the 
plant state which could result in an improved estimate |23|. In the certainty equivalent setup, the probing action cannot be 
implemented due to the lack of a dual effect and the resulting control actions will not remain the same. 



C Conditions for Certainty Equivalence 

From the previous discussions, it is clear that a scheduling criterion independent of the past control actions results in no 
dual effect. This result is presented below. 

Corollary 3.3: For the closed-loop system defined by ([T])-([5]), with no exogenous network traffic, the optimal controller, 
with respect to the cost in ([6]), is certainty equivalent if and only if the scheduling decisions are not a function of the applied 
control actions, such as in J13] ). 

Proof: In the proof of Theorem 



3.2 



it is clear from (17) that the optimal control policy g for the system {V,f,g} is 
certainty equivalent. 

To show the necessity of this condition for certainty equivalence, we need to show that if the optimal control signal has the 
form in §Tl\ at time k, then the scheduling policy is not a function of the controls for n < k, for all k. Accordingly, assume 
that the optimal control signal is given by ( 17) for k = N — 1, . . . , n + 1. Then, the optimal cost-to-go is of the form in ( 16), 
at time n + 1 and 



^n+l 



N-l 

E 



E[ti{A T S k+1 B(Q 2 + B T S k+1 B)- 1 B T S k+1 AP k , k + S k+1 R w }\f k ] 



when written out explicitly. We know that the optimal control signal u n is obtained by minimizing ( 15 ) at time n. This control 
signal will have the form in §FI\ for all Q 2 > only if s n+ i is independent of u n , or if the estimation error covariances 
Pk\k, for k = {n + 1, . . . , N — 1}, are not a function of u n . From the result in Theorem 3.1 this is only possible when the 
scheduling policy is not a function of u n . Since this is true for n = 0, . . . , N — 1, the scheduling policy must not be a function 

ofu k -\ m 



Corollary |3.3| provides us with a restriction on the scheduler to guarantee certainty equivalence. Note that the resulting 
closed-loop system is not equivalent to the original problem setup, as shown in Theorem |3.2| 



D. Effect of State-based Schedulers with exogenous network traffic 

In this subsection, we re-analyze the effects of a state-based scheduler on the control loop in the presence of exogenous 
network traffic. Thus, we have n k 7^ and a channel output given by ([5]). Recall from Lemma 2.1 that the network traffic 
indicator n k is correlated to the state of the plant x k . The Certainty Equivalence Principle need not hold for plants where the 
measurement noise is correlated to the process noise (24). To focus on the effect of state-based schedulers on the closed-loop 
system, the results presented in the previous subsection did not include exogenous network traffic. Now, we re-derive some of 



the above results for the system in the presence of exogenous network traffic. 

Lemma 3.4: For the closed-loop system defined by ([T])-([5]), the control signal has a dual effect of order r = 2. 

Proof: The MAC output 5 k ([3| is clearly still a function of the applied controls, through the state-based scheduler outcome. 
Thus, the estimation error covariance Pk\k, in ( [12} , remains a function of the applied controls Uq' 1 . Since Pk\k does not satisfy 
the condition (10) required to have no dual effect, we see that the system Q-Q exhibits a dual effect of order r = 2. 



With the above result, Theorem 3.2 can be easily extended to include the case with exogenous network traffic. However, 



it is not as straightforward to extend Corollary |3.3| When the measurement noise is correlated to the process noise, certainty 



equivalence need not hold. To see why, recall the proof of Theorem 3.2 where we derive a solution of the form V k 



E[x^SkXkW] + $k for the Bellman equation J15J). Now, if Wk is correlated to the variables in the information set I k , 
specifically nj, the minimization with respect to Uk in jl7|) must include the term tr{Sk+iR w }- Then, the optimal controller 



will not have the form shown in ( 17 ), and certainty equivalence will not hold. 

We need to prove that Wk is independent of ng for the Certainty Equivalence Principle to hold, which we do below. 

Corollary 3.5: For the closed-loop system defined by ([T])-([5|, the optimal controller, with respect to the cost criterion ([6]), is 
certainty equivalent if the exogenous network traffic indicator n k is independent of the process noise w k , and, if the scheduling 
decisions are not a function of the applied controls, i.e., if 

7* = /*(**), (20) 



G (l s k , and Clf, is the sigma-algebra generated by the information set I k = {xq,w% 1 ,np 1 }. 



.fc- 



Proof: Note that n k is only correlated to 5% and thus, to the signals w \ from Lemma 2.1 As the process noise is 



i.i.d, nk is independent with respect to ^. A scheduler of the form ( |2Q| ) is not a function of the applied controls, and thus, 
Certainty Equivalence holds. ■ 

IV. Closed-Loop System Architecture 

IN this section, we identify a property of the scheduling policy that results in a simplification of the design of the closed-loop 
system. This enables us to propose a dual predictor architecture for the closed-loop system, which results in a separation 
of the scheduler, observer and controller designs. 

A. Observer Design 

In this section, we propose a structure for the estimator at the controller. Due to the non-linearity of the problem, the estimate 
in general can be hard to compute. 

The estimation error is reset to zero with every transmission, as we send the full state. Consider one such reset instance, a 
time k such that 5k = 1. The state is sent across the network, yk = x k , so the estimate Xk\k = x k . A suitable control signal u k 
is found and applied to the plant, which results in the next state Xk+i. Now, the scheduler can generate one of two outcomes. 
We consider each case, and find an expression for the estimate, below: 

a) 5k+i = 0: We need an estimate of w k . We use the scheduler output as a coarse quantized measurement to generate 
this, as follows: 

£fc+iifc+i = E[asfc+i|I fc+1 ,(Sfc+i = 0] 

= Ax k + Bu k + E[w k \f(w k ) = 0] , (21) 

x k +i\k+i = x k +i - x k+ i\k+i =w k - E[w k \f(w k ) = 0] , 

where, f(w k ) = f{Ax k + Bu k + w k \x k ,u k ). 

b) 5 k +i = 1: The estimation error is zero as Xk+i\k+i = x k +\. 



The transformation to / in (21 ), is not intended to remove the dual effect, but merely serves to remove the known variables 
from the expression. The dual effect has influenced the packet's transmission, i.e., the value of 5 k +i. 

To see this more clearly, we look at the next time instant. Now a signal Uk+i is generated, and applied to the plant. We 
note that Xk+2 = A 2 Xk + ABuk + Biik+i + Awk + Wfc+i- The state Xk+2 is either sent to the controller or not depending on 
the scheduler outcome 5k+2- Again, we look at both cases, and derive an expression for the estimate: 

i) 5k-\-2 = 0: We now need to estimate Awk + w k +i, as the rest is completely known from x^2- We use both scheduler 
outputs 5 k+ \ and 5k+2 to generate an estimate of the unknown variables as 

Xk+2\ k +2 = A 2 x k + ABu k + Buk+i +E[Aw k +w k+1 \f(w k ) = 0,f(Aw k +w k+1 ) = 0] , 
Xk+2\ k +2 = Aw k + Wfc + i - E[Aw k + w k+1 \f(w k ) = 0, f(Aw k + Wfc+i) = 0] . 
ii) 5k-\-2 = 1- The estimation error is zero as Xk+2\k+2 = Xk-\-2- 
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Fig. 4. State-based Dual Predictor Architecture: the innovations to the observer serve as input to the scheduler. The resulting setup is certainty equivalent. 
The observer is simple, and computes the MMSE estimate. 



This process can be continued recursively through a non-transmission burst, until finally a measurement is received and the 
estimation error is reset to zero. Thus, the observer computes the estimate at any time fc as 



Xk\k 



A k-r kQ 



k — r k k—r k 

: Tfc + J2 A s "% fe - S +E[^ A s - 1 w k - s \f k ,..J Tk+1 =0], 



Sk = 1, 
4 = 0, 



(22) 



S = l 



8=1 



where T& is the time index of the last received measurement at time fc, as defined in (|8j), and the argument to the function f t 
is given by the term ^ s =i* A s ~ 1 w t - S - 



B. State-based Scheduler Design: Symmetric Schedulers 

The computation of the term EE s =i~ fc A s ~ l Wk- s \fk, ••> fr k +i — 0], for a burst of non-transmissions of length greater than 
one, makes the estimate given in ( [22] ) hard to evaluate. This is because the quantized noise is not Gaussian. As a sub-optimal, 
but simplified approach, consider the scheduling criterion given by any symmetric map / s ^ m (r) = / s ^ m (— r) with 



k-r k - 



lk 



f sym { J2 A^Wk-s) • 



(23) 



s=l 



Since r^ is not defined without the MAC output 5k in ([5]), we replace it with Tfc_i, which is also a measure of the non- 
transmission burst. Choosing the scheduler in this manner results in a zero mean estimate from the quantized noise when 
there is no transmission. Now, the estimate is easy to compute and the observer can be designed without knowledge of the 
scheduling policy. Also, a certainty equivalent control can be applied. This observation is summarized below, and is used to 



design the scheduler presented in Section IV-C 



Proposition 4.1: For the closed-loop system defined by ([T])-(|6]), the use of the symmetric scheduling policy defined in (23) 
implies that certainty equivalence holds, and it also results in separation in design, between the estimator and scheduler. 



C. The Dual Predictor Architecture 

In this section, we examine closed-loop design of the complete system, including scheduler, observer and controller. From 



the results of Lemma 3.4 and Proposition 4.1 it is clear that the scheduler, observer and controller designs are coupled. It is 
not possible to design the optimal scheduling policy independently and combine it with a certainty equivalent controller and 
optimal observer to get the overall optimal closed-loop system. At the same time, solving for the jointly optimal scheduler, 
observer and controller is a hard problem. 

Thus, we propose an architecture, shown in Fig. |4j for a design of the state-based scheduler, and the corresponding optimal 
controller and observer. There are two estimators in this architecture, and hence, we call it a dual predictor architecture (29). 
This architecture has been referred to previously in [30], in the context of mobile networks. The scheduler, observer and 
controller blocks are described below. 
Scheduler (<S): The scheduler output jk is given by 



7fe = f{x k ,Xk\, k _ 1 ) 



\xk -EfciTfc-xr > e > 

otherwise. 



(24) 



Here, Xk\T k _ 1 is the estimate at the controller at time fc if the current packet is not scheduled for transmission. To realize such a 
scheduling policy, the observer must be replicated within the scheduler, and for the observer to be able to subtract the applied 
control, the controller must also be replicated within the scheduler. An explicit ACK is required to realize this information 
pattern. 
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Observer (O): The input to the observer is the signal y^ = SkXk. The observer generates the estimate Xk\k as given by 

x k \k = SkXk\r k + SkXk • (25) 

Recall that Sk = 1 — Sk takes a value 1 when the packet is not transmitted. In such a case, the estimate is given by Xk\T k , a 
model based prediction from the last received data packet at time r^. This estimate is given by 



Xk\T k = Axk-i\k-i + Buk-i . 
Controller (C): The controller generates the signal Uk based on the estimate alone, as given by 

Uk = —LkXk\k , 

where Lk is defined in ( fT7) . 

Note that the scheduling criterion described in ( [24] ) can be rewritten as 

|2 



(26) 

(27) 



Ffc - aJfciTfc- 



= 


|AEfc- 


_1 +5M fc _i 


H-^fc- 


-l - 


- Axk-i\k- 


-1 


= 


|Ar fc _ 


-l|fc-l +^fc- 


-ii 2 = 


|£* 


I 2 





#^fc- 



Here, we use Xk\r k _ 1 as r k is not defined without £&. The scheduling criterion |^fc|^ fc _! | 2 < e captures the per-sample variance 
of the estimation error, when no transmission is scheduled. Taking expectations on both sides, we get ti{Pk\r k _ 1 } < e. The 
scheduler attempts to threshold the variance of the estimation error, but this cannot be guaranteed in a network with multiple 



traffic sources. Also, note that the scheduling policy is a symmetric function of its arguments, as in Proposition 4.1 We now 
state the main result of this section. 



Theorem 4.2: For the closed-loop system given by the plant in ([!), the state-based dual predictor architecture in (24)-(27), 
and the cost criterion in ([6]), it holds that 

i. The estimate ( [25) minimizes the mean- squared estimation error. 

ii. The control signal does not have a dual effect. 

iii. The Certainty Equivalence Principle holds and the optimal control law is given by ( [27] ). 

iv. The LQG cost is given by 

N-l 

Jdp = xgSoxo + ti{S Po} + Yl tr (Sn+ A + (Ll (Q 2 + B T S n+1 B)L n )P Mn } , (28) 

where Pk\k is the error covariance of the estimate at the observer, with Sn = Qo and Sk obtained by backward iteration 

of £5). 

Proof: Evaluating the expression E[x/ c |I fc ], we get 



E[x k \I 



Xki 

E[A k ~ Tk c 



YZZFiA'-iBuk-t + A'-iwk-i) 



1,6* +1 =0,y r 



>u 



h = 1, 

4 = 0. 



Due to the use of a symmetric scheduling policy ( [23] ), we know that 

k — T k k — T k 

A k ~ Tk x Tk + Y, A n - x Bu k . n = Ax k - llk -! + Bu k . x , E[£) A^W-A^+i = 0] = 

n=l £=1 

We use the above equations to obtain 

E[x k \I C k ] ~- 




fc-i? 



$k 



1, 

0. 



Thus, the estimate in ([25) is the MMSE estimate II3TI 

The error covariance at the estimator is given by (12), where, from ( [24) and ([3), it is clear that the scheduler outcome jk 
and the MAC output Sk do not depend on the applied controls nj -1 . Thus, the error covariance satisfies the definition in (10), 
and the control signal in this architecture does not have a dual effect. 



From the above conclusion, note that the scheduling policy in ( [24) is of the form ( [20) . Thus, from Corollar y 1 3.5 [ we know 
that the optimal controller for this setup is certainty equivalent. Then, the optimal control signal is given by (|17|), which has 



the same form as the controller in this architecture (27). The expression for the control cost remains the same as in the case 



with partial state information, and is given by ( [28) . ■ 

Thus, the dual predictor architecture results in a simplified design for the closed-loop system, which is optimal within its 



class (as seen from Corollaries 3.3 and 3.5), but not optimal among all possible architectures. 



V. Extensions and Discussions 

IN this section, we extend the above results to an output-based system. We also identify the existence of a dual effect when 
the cost function penalizes network usage and when the transmission, with a state-based scheduler, occurs over limited 
data-rate channels. Finally, we discuss the dual effect property that we have encountered in this problem with respect to other 
NCS architectures. 

A. Measurement-based Scheduler 

We now consider a system without full state information, but with co-located measurements. We show that by placing an 
optimal observer, a Kalman Filter (KF) at the sensor, to estimate the state of the linear plant, and basing the scheduler decisions 
on this estimate, instead of on the measurement, we are able to re-establish the same problem formulation as before. 

Consider a linear plant with a state z k , and a measurement m k given by 

Zk+i = Az k + Bu k + w Zik , m k = Cz k + v Zik , (29) 

where w z ^ k is i.i.d. zero-mean Gaussian with covariance matrix R w , z . The initial state zq is zero-mean Gaussian with covariance 
matrix R Zj o- Also, the measurement m G M m and the matrix C G M. nxm . The measurement noise v Zjk is a zero mean i.i.d 
Gaussian process with covariance matrix R VyZ G ]R m:cm , and it is independent of w Zyk . 

We can always place a Kalman Filter at the sensor node, which receives every measurement m k from the sensor and updates 
its estimate (z klk ) as 

Zk\ k = Azk-i\ k -i + Bu k -i + K fik e k , (30) 

where, Kf jk denotes the gain of the Kalman filter and e k denotes the innovation in the measurement. The innovation can be 
shown to be Gaussian with zero-mean and covariance R e , k - The error co variances for the predicted estimate and the filtered 
estimate are denoted P% and P fc % respectively. These terms are given by 

e k = m k - C(AzLi ]k -i + Bu k -{) , K ftk = P k C T R~l , R e , k = CP s k C T + R v , z , 



P k — AP k _ 1 \ k _ 1 A. -\- R w ^ z , ±h\h — P k ~ -Kf,kRe,k-&f, k • 

Now, if we use the estimate to define a new state, such that x k = z klk , we have a linear plant disturbed by i.i.d Gaussian 



process noise w k = Kf^ k e k . Thus, we have re-established the problem setup from section II-A and the results from before 
can be applied to this plant. Note that the scheduler is now defined with respect to the estimate z k]k and not the measurements 
m k . 

B. Penalizing Network Usage 



We have shown, in the proofs of Theorem 3.1 and Theorem 3.2 that the applied controls play a significant role in a state 



based scheduler and cannot be removed from the scheduler inputs to create an equivalent setup without a dual effect. However, 
the minimizing solution to a cost criterion can render the effect of the applied controls redundant. To see an example of this, 
consider the problem of finding the jointly optimal scheduler-controller pair for the classical LQG cost criterion in ([6]). Since 
there is no penalty on using the network, the optimal scheduler policy is to transmit all the time. Now, the structure of the 
closed-loop system does not resemble the one presented in Theorem |3.1| and consequently, that result does not hold. In this 
scenario, there is no incentive for the controller to influence the transmissions and the jointly optimal scheduler-controller pair 
(Z 1 , g 1 ) is given by 

f:6l = l\/k, g t :ul = -L k x k \fk, (31) 



where L k is given in ( 17 ). Note that in the rest of this paper, we do not consider finding the jointly optimal scheduler-controller 
pair, as the use of a contention-based MAC does not permit us to choose the schedule sequence. 
Now, consider a cost criterion which penalizes the use of the network, such as 



Ja = min E 



- X 1 



N-l 



Qo%N + JZ ( x ^Q lX s + u ^Q^ u s + A<J £ 



(32) 



where Qo,Qi and Q2 are positive definite weighting matrices and A > is the cost of using the network. The optimal 
state-based scheduling policy chooses a schedule in relation to the penalty A, such that the average network use, i.e., E [<$&], 
decreases as A increases. Thus, we state the following result. 

Lemma 5.1: For the closed-loop system defined by ([T])-([5]), with no exogenous network traffic, the control signals derived 



from the jointly optimal scheduler-controller pair, which minimize the cost criterion in ( [32] ), exhibit a dual effect of order 
r = 2. 



Proof: It is easy to show that the scheduler-controller pair (f\g ) does not minimize the cost in (32). Now, the scheduler 
uses the policy in ^ to select packets to send across the network. Thus, the closed-loop system has the same structure as in 
Theorem |3.1 1 and there is a dual effect of order r = 2 for any control signal in this setup. ■ 
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Fig. 5. The estimate is not influenced by the applied controls in (a) and (b), with knowledge of the applied controls. In contrast, the applied controls cannot 
be removed from the decision process in (c). 



This provides the controller an incentive to modify the transmission outcome. As a result, the optimal scheduler and controller 



designs in this problem are coupled. Using the results of Lemma |3.4| the above results can be extended to include the effect 
of exogenous network traffic. 



C. Using a rate constrained channel 

Our proof of the dual effect in Theorem 3.1 relies on the asymmetry in the resolution of the received information; the full 
state is sent with a transmission and only a single-bit quantized encoding of the state is sent when there is no transmission. 
However, data channels are generally rate-constrained, and a full state can never be sent. If the encoder-decoder pair on the 
sensor link uses R bits of information, the estimation error at the controller can be written as Xk\k = 5 k • (xk — E[#fc|I fc , 5k = 
1]) + $k ' ( x k — E[#fc|I fe , 5k = 0]), in place of dllb. Again, note that the estimation error is a function of 5k due to the 
asymmetry in the number of bits in the received information with and without a transmission. Also, the applied controls 
cannot be removed from the above expression, unless the estimation error with and without a transmission result in the same 
expression, i.e., Xk — E[^|I fc , 5k = 1] = Xk — E[xfc|I fe ,5k = 0]. Hence, there is a dual effect with a state based scheduler, 
even while using a rate constrained channel for transmission. 



D. Relation to Other NCS Architectures 

The dual effect and certainty equivalence properties have been noted previously in other NCS problems. We discuss these 
occurrences and the connections to our problem setup below. 



Packet Drops over a Lossy Network: Packet drops in a lossy network are not influenced by the applied controls (Fig. |5(a)| ). 
Hence, certainty equivalence holds, when there are packet drops on the sensor link [19]. However, when there are packet drops 
on the actuator link, separation holds only if there is an ACK of packets received or lost ifTHl . 

Importance of Side Information: In any NCS problem, the classical information pattern must be reconstructed for the 
Certainty Equivalence Principle to hold l32ll . This may require one or more explicit side information channels to convey 
acknowledgements of received packets back to the transmitters (33), (18) . 

Encoder Design over Limited Data Rate Channels: In this problem, the encoder output is the only measurement available 
across the channel, which always contains the same number of information bits. Thus, the applied controls can be removed 
from the decision process, and they do not influence the estimation error, as shown in (20). 

Event-based Systems: The results we have encountered in this paper show that the applied controls can push the state across 
the scheduler threshold, and influence the transmission outcome, as illustrated in Fig |3(b)| This is a consequence of the unequal 
information in the measurement yk, with and without a transmission. 

A similar problem with a cost function such as ( [32] ), has been dealt with in (34), [35]. They use a transformation similar to 
the one presented for the encoder design problem in (2D . There are, however, subtleties in defining an equivalence class for 
a state-based scheduler. Using an equivalent scheduler need not result in an equivalent system, as shown in Theorem |3.2| 



The dual effect is visible in any control signal applied to the plant, not just the optimal one, as the control signal will always 
influence the estimation error, irrespective of whether it has been designed to do so or not. Also, the dual effect exists despite 
knowledge of the applied controls at the scheduler, and knowledge of the scheduling decisions at the controller. In this context, 
the dual effect can be best explained as a coupling between the control and scheduling policies. 



VI. Examples 

WE present three examples in this section. The first example describes the problem setup, and illustrates the motivation 
for the problem. The second example illustrates the results of Theorem |3.1| and Theorem 3J2 which identify the dual 
role of the applied controls towards the information available to the controller. The third example illustrates the dual predictor 
architecture and provides an example of network- aware event triggering. 



TABLE I 
A COMPARISON OF CONTROL COSTS WITH ( J S s) AND WITHOUT (Jqn) A STATE-BASED SCHEDULER IN THE CLOSED-LOOP, FROM EXAMPLe |VI-A| 



Plant Type 


-p[Tl\ 


p[T2\ 


p[T3\ 




45.3074 
23.5785 


10.0028 
8.3489 


6.1213 
5.3803 
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24 



Slot Numbers 
44 54 



-— State 

— Scheduler Threshold 
• Scheduled Transmission 
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Sampling Instants 



Fig. 6. The state and the control signal with the channel use pattern: the red circles denote transmission requests, the white circles denote 
MAC re-transmission attempts, and the green circles denote transmission success. Note that the requested bound on the state, which is 
marked with a dotted line, is sometimes exceeded due to network traffic. Also, the control signal corresponds closely to the state only when 
there is a successful transmission. 



A. An Example of a Multiple Access NCS 

This example illustrates the role of a state-based scheduler in our problem formulation in Section II-A where a number 
of closed-loop systems share a contention-based multiple access network on the sensor link. We use a p-persistent CSMA 
protocol in the MAC. The observer and controller are chosen for simplicity of design, not as optimizers of any cost. We look 
at the performance of this network of control loops, with and without the state-based scheduler. 

Example We consider a heterogenous network of 20 scalar plants, indexed by j G {1, . . . , 20}. There are three different types 
of plants, V [T1 \V [T2] and V [T3 K given by 



r U) _ n [i\AJ) 



a l >xl 



.0) 



w 



U) 



(33) 



where a® G {1, 0.75, 0.5}, and B$ G {1, 1.5, 2}, for the plant V [Ti] . The systems numbered j G {1, . . . , 6} are of type V [T1 \ 
j G {7, . . . , 13} are of type V^ T2 ^ and j G {14, . . . , 20} are of type p[ T3 l. The plants are sampled with different periods given 



,UY 



> e 



U) 



by tW G {10, 20, 25}, for the different types of plants, respectively. The state-based scheduler uses the criterion 
A p-persistent MAC, with synchronized slots, which permits three retransmissions is used. The persistence probability is given 
by Pa T , where r denotes the retransmission index, and pa = {1, 0.75, 0.5} for r G {1, . . . , 3}. The LQG criterion in (|6|), with 
N = 10 and Qo = Qi = Q2 = 1 is used to design a certainty equivalent controller $FJ\ as an ad hoc policy, not an optimal 
one, as we know from Corollary |3. 5 1 The observer calculates a simple estimate as given by (|25])-([26). 

We look at the performance of a closed-loop system in this network without a state-based scheduler, i.e., when e^ = for 
all j. The cost of controlling the plants in the contention-based network, is denoted J^, and the values are listed in Table m| 
We compare these to the costs obtained with a state-based scheduler in the closed-loop system, denoted JJQ, when e^ = 2.5. 
There is a marked improvement with a state-based scheduler in the closed-loop. Fig [6] depicts the state and the control signal 
for the first plant in this network. The above improvement is obtained due to fewer collisions in the contention-based MAC. 
The non-zero scheduling threshold reduces the traffic in the network, and increases the probability of a successful transmission 
for all the plants in the network. 
B. A 2 -Step Horizon Example 

We now look at a simple example to see the computational difficulties in identifying optimal estimates and controls for a 



system with a state-based scheduler in the closed-loop. We also show that for an equivalent scheduler such as / in Section |III-B 
which renders the control signal free of a dual effect, the entire plant is altered, so the equivalence construction does not work. 

axk + buk + Wk, with a, b G R and xo,Wk ~ A/"(0, 1). The scheduling 



Example Consider a scalar plant, given by Xk+i 
law is given by 



fa = 



Xk > 0.5 
otherwise 



Our aim is to find both the optimal controller, with dual effect, and the certainty equivalent controller for the equivalent 
scheduler and show that these result in different control actions for the same scheduling sequence. The controllers are designed 
to minimize the LQG cost ([6]), for a horizon of two steps, i.e., N = 2, and with Qq- ) Qi- ) Q 2 € M. We first derive the 
optimal controller with dual effect. Then, for the same schedule, we define the certainty equivalent controller, assuming that 



an equivalent scheduler of the form / in ([13]) has been designed. We compare the resulting control actions, and comment on 
the differences. 



Estimator: The estimates x |o and x±\i are obtained using (22). The estimation error covariances P \ and P±\i are presented in 



the Appendix. Since the estimation error is non-Gaussian, we need to derive the probability density functions of the estimation 
errors at each time instant. This makes the computation of the estimation errors and the error covariances hard. 
Optimal Controller: To solve for the optimal control signals, we use V\ and Vo from ( p3] >. The complete derivations of V\ 
and Vq are presented in [36]. We find the control signal u\ that minimizes V± 9 and get 

abQo A 
ui = —pz . 2r . a?i|i • (34) 

Then, to find uo, we take a partial derivative of the expression for Vq with respect to uq and get 

S = 2MQ2 + b2Sl) + 2 ^° abSl + ^%- • ik ( E[Pl|l|I ° C] ) = ° ' (35) 

This can be simplified using the expression for P\u. When Sq = 1, we have 

cfV r/ 2 2 fo 2 

- — = 2u {Q 2 + b 2 Si) + 2x \ abSi - 7^ — , ° 2 ^ b(w , m ax ~ wo) 2 0w o (wo,max) = , 

where wo yrnax = 0.5 — axo — buo. The final equation is obtained using Leibnitz rule. For the case when Sq — 0, we have 

&V r/ 2 2 fo 2 

— — = 2uq(Q 2 + b 2 Sx) + 2xQ\QdbS 1 - — — ° b(e max - e 5 o) 2 (j) e5Q (e rnax ) = , 

where e max — 0.5 — buo and again, Leibnitz rule was used. Solving these equations give the optimal ^0 for 5q = 1 and 0, 

respectively. 

CE Controller: For the same scheduler outcomes So, Si obtained through an equivalent scheduler which has no dual effect, 

the certainty equivalent controller gives us the control signals 

AbQo A 

Q2 + b 2 Q „„ 

AbS, A (36) 

Uo = -q 2 -t^s; Xo10 ' 



Note that the u\ is found by minimizing Vi, which results in the same expression as for the optimal controller ( [34] ). However, 
^0 for the CE controller is obtained by solving the equation 

2u (Q 2 + 6 2 #i) + 2x \ Q abS! = . (37) 



Discussion: A comparison of the control signals for the CE controller ( [36] ) with u\ and uq obtained in ( [34] ) and ( [35] ), shows 
that the signal u\ remains the same. However, uq is different, and displays a dual effect in the optimal controller. From ( [37] ), 
it is clear that the additional term in ( [35] ) alters the solution for the optimal controller. 

This observation can be explained as follows. In a controller with a dual effect, the control signal can be chosen to probe 
the plant state in order to improve the quality of the estimate. However, there is no motive in improving the estimate in a 
one- step optimization process. Thus, u\ is the same for both controllers. When the optimization is performed over two steps, 
a probing effect in the first step can improve the estimate and the corresponding control applied in the next step. Thus, ^o is 
different in the optimal controller for a state-based scheduler. 

This example shows us that even the same schedule can result in a different control sequence for a system without a dual 
effect. Thus, an equivalent construction for the scheduler does not result in an equivalent system in our setup. 

C. An Example of the Dual Predictor Architecture 

In this example, we present the dual predictor architecture, as applied to a shared network. We tune the threshold of the 
state-based scheduling law to probabilistically guarantee an achievable control performance, given the traffic over the network. 
We use a homogenous network in this example to simplify the comparison of control cost versus the scheduling threshold. 



Example We consider a shared network of 20 scalar plants, indexed by j G {!,..., 20} and given by (33), where a^ = 1 



? (i) 



and R)i ) = 1 for all j. The plants are sampled with a period given by T = 10. The innovations-based scheduler uses a 



similar criterion to (24), where e is the threshold of the scheduler. A p-persistent MAC, with synchronized slots, which permits 



LQG Cost versus Epsilon 




Epsilon 



Fig. 7. The control cost Jdp versus the scheduler threshold e. For low thresholds, the high traffic in the network causes collisions, and a high Jdp. Very 
high values of e result in an under-utilized network, and a high Jdp due to insufficient transmissions. 



three retransmissions is used. The persistence probability is given by p a 

( r ) _ 



(r) 



where r denotes the retransmission index and 



Pa ' — {1, 0.75, 0.5} for r € {1, ... , 3}. The LQG criterion in ([6]), with N = 10 and Qo = Qi = Q2 = 1 is used to design 
the optimal certainty equivalent controller (fTTl). The observer calculates the MMSE estimate given by ([25])-([26). 



The effect of varying e on the control cost is shown in Fig. [7] For very high values of e, the network is under-utilized, and 
almost all the transmissions are successful. However, the control cost is high as the number of transmissions is low. As we 
decrease e, the control cost initially decreases due to increased use of the network. However, for very low values of e, the 
network is over-utilized and this results in collisions. Thus, the control cost increases again, due to dropped packets. 

Fig. [8] depicts the state and control signal of the first plant obtained from our simulation, for the best value of e picked from 
the above plot. Note that the estimation error is bounded, with a probability of 0.94, by the scheduling threshold, for the value 



,(i) 



3.5, and the resulting control cost is Jdp = 27.9235. 



It is interesting to note, in Fig. [7] that the cost function is quite flat. Thus, it is not very important to use the optimal 
scheduling threshold e. 
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VIII. Conclusions and Future Work 

THIS paper investigates the effects of a state-based scheduler on the design of a closed loop system. We find that the 
optimal controller for a NCS with a state-based (or measurement-based) scheduler is, in general, difficult to find. This is 
due to the dual effect of the control signals in the given setup, wherein the controller has an incentive to push the state past the 
scheduler threshold and modify the estimation error across the network. This implies that the optimal scheduler, observer and 
controller designs are coupled. However, we identify a dual predictor architecture, which results in the desired separation in 
design of the scheduler, observer and controller. The scheduling function in this architecture is constrained to be a symmetric 
function of its arguments, such that the resulting schedule is not a function of the past applied controls. 
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Fig. 8. The estimation error, state and control signal with the channel use pattern. Note that the requested bound on the predicted estimation error, which is 
marked with a dotted line, is rarely exceeded. Also, the control signal corresponds closely to the state only when there is a successful transmission. 
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Analyzing the performance of a network of systems using the dual predictor architecture is a challenging direction of work, 
for the future. Identifying interesting, if not optimal, control policies in the more general case of state-based schedulers with a 
dual effect, has been left for the future. A complete extension to include distributed measurements of the state is another task 
for the future. 

Appendix 
Derivation of the 2-Step Horizon Example 



Here are the expressions for the estimation error covariances in Example |VI-B For a detailed derivation, refer l36l . 
The estimation error covariance at time k = is given by 

f Rx =E[Oo -x 6 o) 2 \xo < 0.5] 

Polo = < ~ where, />o.5-^ (38) 

[Rxo, ^o = 0, = / x </> Xso (x + xso)dx , 



where x§q := E[£ |^o < 0.5] = J' x<j) X80 (x)dx, (j) X50 is the conditional probability distribution function (pdf) of xq, 
conditioned on xq < 0.5. Thus, (j) X50 (x) = (j) Xo (x)/Pr(xo < 0.5), where (f) Xo is the pdf of xq. The probability of a non- 
transmission is given by Pr(xo < 0.5) = /_' (j) XQ (x)dx. 

Let us denote ei as the unknown part of x\ before y\ is received: 



ei= U, *o = l, and Ue) U^\ *o = l, 

[axo + wo, S = 0, [0e 5o ( e )> ^o = 0, 

where, <j) e is the pdf of e\, (j) Wo is the pdf of wq and ^e S o( e ) — J J ^aoO^O^woC 6 — ax)dx. We denote e\ as the error in 
estimating e\ after y\ arrives, and e§o = E[axo + wq\xq < 0.5, ax^ -\- wq < 0.5 — buo]. Now, the estimation error variance 
Pi|i is given by 



where R ei = E[ef |£i = 0] is given by 
Note that increasing i^o will decrease R ei 



,.-.■0-^-0 ^n(^o) . ^ 

J-oo 6 Pr( ei <0.5-6n )^ e ' °° ~ U ' 
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